TITLE OF THE INVENTION 

Image Processing Circuit 
BACKGROUND OF THE INVENTION 
Field of the Invention 

5 The present invention relates to an image processing circuit for 

image-processing image data picked up with an image pickup device such as a digital still 
camera. 

Description of the Background Art 

Fig. 13 schematically illustrates the structure of a general digital still camera 

10 100. As shown in Fig. 13, an image signal picked up with an image pickup device 105 
such as a loaded CCD or CMOS sensor is A/D converted to a digital image signal and 
thereafter captured by an image processing part 106, to be subjected to various image 
processing such as pixel interpolation, color space conversion and contour correction. 
The image data subjected to the image processing is displayed on a finder of a liquid 

15 crystal monitor 109 or the like, stored in a memory card 110 or output to an external 
device such as a personal computer through an interface 111. Referring to Fig. 13, 
numeral 101 denotes an optical lens, numeral 102 denotes a color correction filter, 
numeral 103 denotes an optical LPF (low-pass filter), numeral 104 denotes a color filter 
array, and numeral 107 denotes a driving part driving/controlling the image pickup 

20 device 105 etc. 

As shown in Fig. 14, the image signal picked up with the image pickup device 
105 is converted to a digital image signal (CCD data) and thereafter temporarily stored in 
a raw image data buffer 108a of a built-in memory 180 (step 100). Then, a real-time 
processing unit (hereinafter abbreviated as RPU) 120 formed by hardware provided in the 

25 aforementioned image processing part 106 reads the raw image data stored in the raw 



image data buffer 108a, executes the aforementioned image processing such as pixel 
interpolation, color space conversion and edge enhancement in real time, and outputs and 
stores the processed data to and in a processed data buffer 108b (step 101). Then, a 
CPU (central processing unit) 121 reads the processed data from the processed data 
buffer 108b at an instructed timing, compresses the processed data with a temporary 
storage buffer 108c by software processing in the JPEG (joint photographic experts 
group) system or the like, and stores the compressed data in a storage medium 122 such 
as the aforementioned memory card 110. 

While the recent digital still camera is miniaturized and reduced in weight, the 
aforementioned built-in memory 108 is formed by a DRAM (dynamic random access 
memory) or an SDRAM (synchronous DRAM) and the buffer areas 108a to 108c used in 
the built-in memory 108 are increased in scale to disadvantageously increase the cost for 
a chip as well as power consumption. 

Either an interlacing (interlace scanning) system reading two fields forming a 
frame, i.e., an even field consisting of only even lines and an odd field consisting of only 
odd lines, at absolutely different temporal timings or a progressive (sequential scanning) 
sequentially reading respective lines is employed for the aforementioned image pickup 
device 105. If the interlacing system is employed, three to five lines including a current 
line and a plurality of precedent and subsequent lines must be simultaneously processed 
when the RPU 120 executes pixel interpolation, for example, and hence it is necessary to 
capture the even field after storing the odd field in the raw image data buffer 108 a for 
thereafter converting both fields to progressive data and outputting the same to the RPU 
120. Therefore, reduction of the scale of the buffer areas 108a to 108c of the built-in 
memory 108 is limited. 
SUMMARY OF THE INVENTION 



According to a first aspect of the present invention, an image processing circuit 
processing raw image data picked up with an image pickup device comprises 
compression means compressing raw image data obtained by A/D converting the raw 
image data, a buffer part temporarily storing compressed data transferred from the 
compression means, expansion means reading the compressed data from the buffer part 
and expanding the same and an image processing part executing image processing on 
expanded data transferred from the expansion means. 

The image processing circuit according to the first aspect temporarily 
compresses the raw image data and stores the same in the buffer part for performing 
image processing by reading the compressed data while expanding the same, whereby the 
scale of buffer areas of a memory can be reduced. Thus, it is possible to minimize the 
memory in response to the number of pixels of the image pickup device such as a CCD 
for optimizing the buffer areas thereof. 

According to a second aspect of the present invention, the image pickup device 
is driven by an interlacing system reading an odd field consisting of only odd lines and an 
even field consisting of only even lines forming a frame at temporally different timings, 
the buffer part stores the compressed data of a first field formed by either the odd field or 
the even field, and the image processing part reads the first field stored in the buffer part 
in synchronization with entry of a second field formed by the remaining field and 
executes real-time image processing on the first and second fields. 

According to the second aspect, the image processing part can execute 
real-time image processing on the first and second fields in the image processing part 
during a reading period for the second field, whereby the real-time image processing can 
be efficiently ended at the moment when reading of the second field is ended. Therefore, 
a picked-up image can be displayed on a finder substantially without a delay, for 



example. 

According to a third aspect of the present invention, data transfer between the 
compression means and the buffer part, and data transfer between the expansion means 
and the buffer part are controlled by a direct memory access system. 

According to the third aspect, data can be directly transferred between the 
compression means and the buffer part as well as between the expansion means and the 
buffer part without through a CPU, whereby the data can be processed at a high speed. 
Particularly when the image processing circuit according to the second aspect is 
employed and the image pickup device is driven by the interlacing system, the image 
processing part can efficiently perform real-time image processing in the reading period 
for the second field while transferring the first field stored in the buffer part to the 
expansion means by a DMA system at a high speed and expanding the same and the 
buffer part may not temporarily store the second field, whereby the scale of buffer areas 
on the memory can be reduced. 

According to a fourth aspect of the present invention, the image processing 
circuit further comprises division means dividing the digital image data into a plurality of 
blocks and outputting the same to the compression means, and the compression means 
and the expansion means execute compression and expansion in units of the blocks. 

When a defective pixel or the like is detected, only a block including this 
defective pixel can be corrected or replaced with another normal block according to the 
fourth aspect, whereby influence by the defective pixel can be efficiently suppressed to 
the minimum. 

According to a fifth aspect of the present invention, the image processing 
circuit further comprises means detecting a block including previously specified defective 
pixel data among the expanded data expanded by the expansion means and outputting a 



block obtained by correcting the defective pixel data to the compression means. 

According to the fifth aspect, previously specified defective pixel data is 
detected and corrected in units of blocks, whereby it is possible to efficiently correct a 
defective block for displaying and preserving a high-definition image at a high speed. 

According to a sixth aspect of the present invention, the image processing 
circuit further comprises defect inspection/correction means performing a defect 
inspection before outputting the expanded data expanded by the expansion means to the 
image processing part for replacing a block having detected defective pixel data with a 
normal block and outputting the same to the compression means. 

According to the sixth aspect, defective pixel data is detected in units of blocks 
for replacing a defective block with a normal block, whereby it is possible to efficiently 
correct the defective block for displaying and preserving a high-definition image at a high 
speed. 

According to a seventh aspect of the present invention, the division means 
divides the digital image data into a plurality of blocks in units of lines. 

According to the seventh aspect, it is possible to detect a defect in units of lines 
for efficiently correcting the defect and providing a high-definition object image. 

According to an eighth aspect of the present invention, the image processing 
circuit further comprises difference calculation means calculating the difference between 
pixel values of the digital image data and outputting the difference to the compression 
means before compressing the digital image data in the compression means. 

According to the eighth aspect, the image data can be efficiently coded and 
stored in a buffer area, whereby the memory can be minimized in response to the number 
of pixels of the image pickup device such as a CCD or the like and the buffer area thereof 
can be optimized. 



According to a ninth aspect of the present invention, the difference calculation 
means calculates the difference between the values of pixels adjacent to each other along 
the time base. 

According to a tenth aspect of the present invention, the difference calculation 
means calculates the difference between the values of alternate pixels along the time 
base. 

According to an eleventh aspect of the present invention, the difference 
calculation means calculates the difference between the values of vertically adjacent two 
pixels of two lines of the digital image data. 

According to a twelfth aspect of the present invention, the difference 
calculation means calculates the difference between the values of vertically adjacent two 
pixels of alternate lines of the digital image data. 

According to the ninth to twelfth aspects, image data can be efficiently 
compression-coded. 

According to a thirteenth aspect of the present invention, the difference 
calculation means according to either the eleventh or twelfth aspect is selected in 
response to the driving system for the image pickup device. 

According to the thirteenth aspect, differential compression of the values of 
pixels of the same color component can be executed to be suitable to the arrangement of a 
color filter array of the image pickup device. 

Accordingly, an object of the present invention is to provide an image 
processing circuit capable of reducing the scale of buffer areas in a memory for 
remarkably reducing the cost for the memory as well as power consumption. 

The foregoing and other objects, features, aspects and advantages of the present 
invention will become more apparent from the following detailed description of the 
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present invention when taken in conjunction with the accompanying drawings. 
BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic block diagram showing the overall structure of a digital 
still camera according to the present invention; 
5 Fig. 2 is a block diagram showing an exemplary structure of a real-time 

processing unit (RPU) according to the present invention; 

Fig. 3 is a schematic block diagram showing the flow of image signal 
processing by an image processing circuit according to an embodiment 1 of the present 
invention; 

10 Fig. 4 is a schematic block diagram showing the flow of image signal 

processing by an image processing circuit according to a modification of the embodiment 
1 of the present invention; 

Fig. 5 is a schematic block diagram showing the flow of image signal 
processing by an image processing circuit according to an embodiment 2 of the present 

15 invention; 

Fig. 6 is a schematic block diagram showing the flow of image signal 
processing by an image processing circuit according to an embodiment 3 of the present 
invention; 

Fig. 7 is a schematic block diagram showing the flow of image signal 
20 processing by an image processing circuit according to an embodiment 4 of the present 
invention; 

Fig. 8 is a schematic block diagram showing the flow of image signal 
processing by an image processing circuit according to an embodiment 5 of the present 
invention; 

25 Fig. 9A is an explanatory diagram showing a differential compression method 
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according to an embodiment 6 of the present invention, and Fig. 9B is a schematic block 
diagram showing an exemplary structure implementing the differential compression; 

Fig. 1 OA is an explanatory diagram showing a differential compression method 
according to an embodiment 7 of the present invention, and Fig. 1 OB is a schematic block 
5 diagram showing an exemplary structure implementing the differential compression; 

Fig. 1 1 A is an explanatory diagram showing a differential compression method 
according to an embodiment 8 of the present invention, and Fig. 1 IB is a schematic block 
diagram showing an exemplary structure implementing the differential compression; 

Fig. 12A is an explanatory diagram showing a differential compression method 
10 according to an embodiment 9 of the present invention, and Fig. 12B is a schematic block 
diagram showing an exemplary structure implementing the differential compression; 

Fig. 13 schematically illustrates an exemplary structure of a general digital still 
camera; and 

Fig. 14 is a block diagram showing an exemplary flow of conventional image 
1 5 signal processing. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Fig. 1 is a schematic block diagram showing the overall structure of a digital 
still camera (image pickup device) 1 according to the present invention. In this digital 
still camera 1, a CCD (charge-coupled device) 12 picks up an image signal of an object 
20 through an optical mechanism 11 formed by an optical system and a diaphragm 
mechanism having an AF (auto-focus) function. At this time, a stroboscope 20 may be 
employed at need for applying light to the object. The image signal of the object as 
picked up is captured in an analog signal processing circuit 13 and A/D converted to a 
digital image signal. 

25 This digital image signal is compressed in an image compression/expansion 
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circuit 9 as described later in detail, and transferred to a main memory 26 to be stored in 
its buffer area. The compressed data is expanded in the image compression/expansion 
circuit 9, and thereafter subjected to prescribed image processing such as pixel 
interpolation, color space conversion, edge enhancement, filtering or culling in a 
5 real-time processing unit (hereinafter abbreviated as RPU) 14 in real time. 

The image signal subjected to real-time image processing as described above 
can be displayed on an LCD 23 serving as a finder, subjected to image processing such as 
JPEG compression in a CPU 17 and thereafter stored in a memory card 27, or output to 
an external device such as a personal computer through an external interface (I/F) 28. 

10 The main memory 26 formed by a DRAM, an SDRAM or an RDRAM (Rambus DRAM) 
is provided with a buffer area temporarily storing data processed in the RPU 14, a buffer 
area temporarily utilized when the CPU 17 executes software processing and the like. 

Referring to Fig. 1, numeral 15 denotes a CCD driving circuit driving the CCD 
12, numeral 16 denotes a timing generator regulating operation timings of the RPU 14 

15 and the CCD driving circuit 15, numeral 18 denotes a PLL oscillation circuit, numeral 19 
denotes a coprocessor which is an auxiliary arithmetic unit for the CPU 17, numeral 20 
denotes a display module, numeral 21 denotes a digital encoder and numeral 22 denotes 
an LCD driving circuit driving the LCD 23. 

The respective elements, i.e., the image compression/expansion circuit 9, the 

20 main memory 26, the external interface 28, the memory card 27, a DMA (direct memory 
access) controller 24 and a JPEG processing part 25 are interconnected with each other 
through a main bus 10, so that data transfer between these elements can be directly 
executed through the main bus 10 under control of the DMA controller 24 without 
control of the CPU 17. Thus, a load on the CPU 17 is reduced while the main memory 

25 26 can be efficiently utilized. 
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When displayed on the LCD (finder) 23, the object image (frame) continuously 
output from the CCD 12 is subjected to real-time image processing for slightly reducing 
the resolution or the like in the RPU 14, and thereafter sequentially displayed on the LCD 
23 as a motion picture. When capturing (picking up) the object image displayed on the 
5 LCD 23, the operator may simply press a pickup button (not shown) provided on the 
digital still camera 1. When the pickup button is pressed, the CPU 17 of the digital still 
camera 1 performs the aforementioned JPEG compression on the object image data for 
storing the compressed data in a storage medium such as the memory card 27 or 
outputting the same to the external device through the external interface 28. 

10 Digital image data output from the analog signal processing circuit 13 may be 

directly output to the RPU 14 to be subjected to the aforementioned real-time image 
processing, or may be temporarily stored in the main memory 26 through the image 
compression/expansion circuit 9 to be thereafter loaded in the RPU 14 and subjected to 
the aforementioned real-time image processing. When partial processing unexecutable 

1 5 in the RPU 1 4 formed by hardware is executed with software in the CPU 1 7, for example, 
the processing speed can be improved to several times to tens of times as compared with 
a case of executing all image processing on software, and power consumption can be 
reduced due to reduction of processing in the CPU 17. 

Either an interlacing (interlace scanning) system reading an even field 

20 consisting of only even lines and an odd field consisting of only odd lines forming a 
frame at temporally different timings or a progressive (sequential scanning) system 
sequentially reading and driving respective lines is selectively employed for the CCD 12 
generally comprising a charge storing part and a charge transfer part therein. While the 
CCD 12 is employed as an image pickup device, the present invention is not restricted to 

25 this but a CMOS sensor having no charge transfer part is also employable. 
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As shown in a block diagram of Fig. 2, the RPU 14 comprises a single pixel 
processing block 14a processing the digital image signal in units of pixels, an 
interpolation and gamma processing block 14b executing pixel interpolation described 
later in detail while correcting gamma characteristics (gamma correction), a color space 
5 conversion and false color suppression block 14c, a spatial filter and coring block 14d 
and a resizing block 14e outputting image data processed in the respective blocks 14a to 
14d to the main bus 10. 

Embodiment 1. 

Fig. 3 is a schematic block diagram showing the flow of image signal 
10 processing by an image processing circuit according to an embodiment 1 of the present 
invention in the digital still camera (image pickup device) 1 having the aforementioned 
structure. 

As shown in Fig. 3, CCD data (raw image data) picked up with the CCD 12 is 
A/D converted to a digital image signal in the aforementioned analog signal processing 

15 circuit 13, thereafter compressed by compression means 31A of the aforementioned 
image compression/expansion circuit 9 by binary arithmetic coding or entropy coding 
such as Huffman coding and transferred to a raw image data buffer 26a provided in the 
main memory 26 through the aforementioned main bus 10 to be temporarily stored 
therein (step 10). The CCD data may be output by either the interlacing system or the 

20 progressive system. 

Then, the compressed data stored in the raw image data buffer 26a is expanded 
in expansion means 3 IB of the aforementioned image compression/expansion circuit 9 
by entropy decoding or the like, so that pixel data thereof is sequentially output to the 
aforementioned RPU 14 (step 11). In parallel with the aforementioned expansion of the 

25 compressed data, the RPU 14 executes image processing on the input pixel data through 
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the single pixel processing block 14a, the interpolation and gamma processing block 14b, 
the color space conversion and false color suppression block 14c, the spatial filter and 
coring block 14d and the resizing block 14e in real time so that the processed data output 
from the aforementioned resizing block 14e is transferred to and stored in a processed 
5 data buffer 26b provided in the main memory 26 in units of frames. While the 
compression means 31A and the expansion means 3 IB execute hardware processing for 
improving the compression/expansion speed in this embodiment, the CPU 17 may 
substitutionally execute the same type of software processing. 

The single pixel processing block 14a performs either multiplication or addition 

10 every pixel of the digital image signal, thereby selectively performing either sequential 
averaging between a plurality of frames or shading correction in a single frame. Such a 
phenomenon that the brightness of the periphery is reduced as compared with the central 
position due to an optical action of a lens or the like when an image of an object or a 
landscape is picked up with the aforementioned CCD 12 is generally referred to as 

15 shading. In the aforementioned shading correction, gain control of the brightness in 
each pixel is executed for relaxing such shading. 

The interpolation and gamma processing block 14b can capture the image data 
temporarily stored in the main memory 26 through the main bus 10 under control of the 
DMA controller 24. The image data can be directly input not only into the single pixel 

20 processing block 14a forming the initial stage of the RPU 14 but also into the next-stage 
interpolation and gamma processing block 14b through the main bus 10, whereby the 
interpolation and gamma processing block 14b can directly process the image data 
subjected to image processing in the CPU 17 without through the single pixel processing 
block 14a. 

25 The color space conversion and false color suppression block 14c has a color 
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space conversion function of converting image data expressed in the RGB-system three 
colors or the YMCG-system four colors to another color space coordinate system when 
an original signal is a color image signal and a false color suppression function of 
performing color suppression (chromatic suppression: false color prevention) on bright 
and dark parts of the image. A YUV coordinate system, a YIQ coordinate system or a 
YC b C r coordinate system employed in the NTSC (national television system committee) 
system or the like may be employed for the converted coordinate system used in the color 
space conversion function. When the YC b C r coordinate system is employed for color 
component conversion, for example, the RGB components thereof are converted to a 
coordinate system of YC b C r components consisting of a brightness signal Y and two 
color difference signals C b and C r . The correlation between the YC b C r components is 
smaller than that between the RGB components, whereby the image size can be 
compressed. 

In general, a dark part of an image is readily influenced by various noises, and 
hence color development is suppressed to the minimum for outputting an image of 
natural picture quality. On the other hand, a bright part of the image is readily 
modulated in response to the characteristics of a CCD picking up the image of the bright 
part or other various hardware components to readily lose white balance, and hence color 
development is suppressed to the minimum for outputting an image of natural picture 
quality. In consideration of these facts, color development in the bright and dark parts 
of the image is suppressed in the false color suppression function. 

After executing the aforementioned image processing, the CPU 17 reads the 
image (frame) from the processed data buffer 26b when the operator of the digital still 
camera 1 presses the pickup button or upon control from the external device connected to 
the external interface 28 and performs software processing such as high-efficiency coding 



of JPEG compression employing DCT (discrete cosine transformation) or wavelet 
transformation, for example, for storing and preserving the processed data in a storage 
medium 30 such as the aforementioned memory card 27 or the external device connected 
to the external interface 28 (step 12). 
5 According to the embodiment 1, as hereinabove described, the raw image data 

is temporarily compressed and stored in the main memory 26 and the compressed data is 
expanded and read so that the RPU 14 executes real-time image processing, whereby the 
scale of buffer areas of the main memory 26 can be reduced. Thus, it is possible to 
minimize the size of the main memory 26 or the like in response to the number of pixels 

1 0 of the CCD 1 2 and optimize the buffer areas. 

As shown in Fig. 4, data transfer between the image compression/expansion 
circuit 9 comprising the compression means 31A and the expansion means 3 IB and the 
main memory 26 as well as between the RPU 14 and the main memory 26 is preferably 
executed not through the CPU 17 but in the DMA system controlled by the DMA 

15 controller 24. 

Embodiment 2. 

The image processing circuit according to the aforementioned embodiment 1 is 
particularly effective for an image pickup device driven by the interlacing system. Fig. 
5 is a block diagram showing the flow of image signal processing in an image processing 
20 circuit according to an embodiment 2 of the present invention applying the image 
processing circuit according to the aforementioned embodiment 1 to a CCD 12 of the 
interlacing system. 

As shown in Fig. 5, either one (hereinafter referred to as a first field) of an odd 
field consisting of only odd lines and an even field consisting of only even lines forming 
25 a frame is AID converted by the aforementioned analog signal processing circuit 13 and 
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thereafter compressed by the compression means 31 A of the aforementioned image 
compression/expansion circuit 9, and the compressed data is directly transferred from the 
image compression/expansion circuit 9 toward a raw image data buffer 26a provided in a 
main memory 26 under control of a DMA controller 24 in the DMA system and stored 
5 therein (step 20). 

After the first field is stored in the raw image data buffer 26a, pixel data of the 
other field (hereinafter referred to as a second field) is AID converted in the 
aforementioned analog signal processing circuit 13 and thereafter sequentially input in an 
RPU 14 while the compressed data of the first field is transferred toward the expansion 

10 means 3 IB of the image compression/expansion circuit 9 from the raw image data buffer 
26a in the DMA system and read. The RPU 14 reads the first field expanded in the 
expansion means 3 IB in synchronization with the second field, executes the 
aforementioned real-time image processing and directly transfers the processed data to a 
processed data buffer 26b of the main memory 26 in the DMA system for making the 

1 5 processed data buffer 26b store the processed data (step 21). 

The image data subjected to the real-time image processing in the RPU 14 and 
stored in the processed data buffer 26c in the aforementioned manner is read by a CPU 17. 
The CPU 17 performs software processing such as JPEG compression using a temporary 
storage data buffer 26c, for storing and preserving the processed data in a storage 

20 medium 30 such as the aforementioned memory card 27 or an external device connected 
to an external interface 28 (step 22). 

Thus, the main memory 26 stores the data (for 1/2 frames) of the first field in a 
compressed state, whereby the scale of buffer areas of the main memory 26 can be 
reduced. Further, data transfer between the compression and expansion means 31 A and 

25 3 IB and the main memory 26 is performed in the DMA system and the RPU 14 can 



perform real-time image processing such as pixel interpolation, color conversion or edge 
enhancement on the first and second fields during a reading period for the second field, 
whereby the real-time image processing can be efficiently ended at the moment when 
reading of the second field is ended. Therefore, a picked-up image can be displayed on 
5 a finder such as an LCD 23 substantially without a time delay. 

When performing the aforementioned real-time image processing on an image 
signal picked up by an image pickup device such as the CCD 12, it is necessary to refer 
to and batch-process pixel data of three to five lines including even and odd lines in pixel 
interpolation, for example. When performing the aforementioned real-time processing 

10 on a CCD of the interlacing system, therefore, line memories for a single field is 
generally prepared in the RPU 1 4. However, the ratio of the line memory occupying the 
overall circuit is too large to suit to miniaturization/size reduction of the device, while the 
number of processible pixels is reduced and limited if the line memory is limited. 
According to this embodiment, the data stored in the main memory 26 is directly 

15 transferred to the RPU 14 in the DMA system to be subjected to real-time image 
processing, whereby no line memory for a single field may be prepared in the RPU 14 
and the number of processible horizontal pixels is hardly limited by the capacity of the 
main memory 26. 

Embodiment 3. 

20 Fig. 6 is a block diagram showing the flow of image signal processing in an 

image processing circuit according to an embodiment 3 of the present invention. 

As shown in Fig. 6, CCD data is AID converted by an analog signal processing 
circuit similar to the aforementioned analog signal processing circuit 13 and thereafter 
input in division means 32 of an image compression/expansion circuit 9, to be divided 

25 into a plurality of blocks in real time (step 30). At this time, a single frame of an input 
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image signal may be divided in units of a plurality of lines or a single line processible in 
an RPU 14, or may be divided into a plurality of tiles through vertical and horizontal 
lines. 

Compression means similar to the aforementioned compression means 31A 
5 compresses such image data by entropy coding or the like in units of blocks, and transfers 
the compressed data to a raw image data buffer 26a of a main memory 26 to be stored 
therein (step 3 1). Alternatively, the image compression/expansion circuit 9 may directly 
transfer the compressed data to the main memory 26 in the aforementioned DMA system. 
The compressed data stored in the raw image data buffer 26a is transferred to expansion 

10 means similar to the aforementioned expansion means 3 IB to be expanded in units of 
blocks, so that pixel data of the expanded data is sequentially output to the RPU 14 and 
subjected to real-time image processing. The data processed in the RPU 14 is output 
and transferred to a processed data buffer 26b of the main memory 26, and stored therein 
in units of frames (step 32). 

15 The CPU 17 reads an image (frame) from the processed data buffer 26b when 

receiving an instruction and executes software processing such as JPEG compression 
using a temporary storage data buffer 26c, for storing and preserving the processed data 
in a storage medium 30 such as a memory card similar to the aforementioned memory 
card 27 or an external device connected to an external interface 28 (step 33). 

20 Thus, the image processing circuit according to the embodiment 3 performs 

compression/expansion in units of blocks. If a defect is found in a certain block, 
therefore, influence of this defect can be suppressed to only this block. 
Embodiment 4. 

Fig. 7 is a schematic block diagram showing the flow of image signal 
25 processing in an image processing circuit according to an embodiment 4 of the present 



invention prepared by improving the aforementioned embodiment 3. The feature of this 
embodiment resides in that the image processing circuit comprises a selector 33 
switching the destination of connection of compression means 31A to either division 
means 32 or a CPU 17 and the CPU 17 executing a defect inspection of a block expanded 
5 in expansion mean 3 IB and defect correction by software processing. 

As shown in Fig. 7, AID converted CCD data is input in the division means 32 
of an image compression/expansion circuit 9, and divided into a plurality of blocks 
having a prescribed bit length in real time (step 40). A divided image signal is input in 
the aforementioned compression means 31 A through the selector 33, compressed in units 

1 0 of blocks by entropy coding or the like, and transferred to and temporarily stored in a raw 
image data buffer 26a of a main memory 26 (step 41). 

Then, the compressed data stored in the raw image data buffer 26a is 
transferred to and expanded in the expansion means 3 IB. A defective pixel of an image 
pickup device such as a CCD 12 is previously recognized through a test in any 

15 fabrication step or a subsequent step, and the address of the defective pixel is written in a 
nonvolatile memory in a digital still camera 1 or the main memory 26. The CPU 17 
refers to the nonvolatile memory or the like with a required software program. When 
reading expanded blocks and detecting a block including defective pixel data, the CPU 17 
corrects the defective pixel data of this block. Then, the CPU 1 7 controls the selector 33 

20 to connect the compression means 31 A with the CPU 17, outputs the corrected block to 
the compression means 3 1 A for transferring and storing the same to and in the raw image 
data buffer 26a of the main memory 26. 

When an image picked up by the CCD 12 includes a defective pixel, the 
defective pixel data may be replaced with normal pixel data precedently input by one 

25 stage as the same color component as the defective pixel, for example. The general 
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CCD 12 has millions of pixels in a single frame, and hence a sufficiently visual image 
can be obtained by replacing the defective pixel by such a relatively simple method. 

When the normal block replacing the defective block is expanded in the 
expansion means 3 IB, the expanded pixel data is sequentially output to the RPU 14 to be 
5 subjected to real-time image processing and thereafter transferred to a processed data 
buffer 26b to be stored therein in units of frames (step 42). Subsequent processing is 
similar to that of the aforementioned step 33 described with reference to the embodiment 
2. 

According to this embodiment, defective pixel data can be corrected in units of 
10 blocks, so that a high-definition image can be displayed on a finder or the like at a high 
speed and preserved in a storage medium. 
Embodiment 5. 

Fig. 8 is a schematic block diagram showing the flow of image signal 
processing in an image processing circuit according to an embodiment 5 of the present 

15 invention prepared by improving the aforementioned embodiment 3. According to this 
embodiment, A/D converted CCD data is input in division means 32A of an image 
compression/expansion circuit 9 and divided every line in real time (step 50). Then, a 
divided image signal is compressed in compression means 31 A every line, and thereafter 
transferred to and stored in a raw image data buffer 26a of a main memory 26 under 

20 control of a CPU 1 7 or a DMA controller 24 (step 51). 

Then, the compressed data stored in the raw image data buffer 26a is 
transferred to expansion means 3 IB to be expanded every line, and the expanded data is 
subjected to defect detection in units of lines in defect detection means 34 of an image 
compression/expansion circuit 9. When the defect detection means 34 detects no defect 

25 caused in compression/expansion on any line, the expanded data is transferred from the 
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expansion means 3 IB to an RPU 14 since a selector 35 connects the expansion means 
3 IB with the RPU 14. When the defect detection means 34 detects a defect on any line, 
the selector 35 is controlled to switch the connection to between the RPU 14 and a line 
memory storing data of a line precedent to the current line or the like, for making the 
5 RPU 14 output the data of the precedent line in place of the current line (step 52). Thus, 
defective pixel data can be detected and corrected in units of lines, so that a 
high-definition image can be displayed on a finder or the like at a high speed and 
preserved in a storage medium. 

While the image processing circuit according to the embodiment 5 divides the 
10 aforementioned CCD data in units of lines and performs defect detection in units of lines, 
the CCD data may alternatively be divided into blocks so that defect detection is 
performed in units of blocks. 

Embodiment 6. 

Before compressing the A/D converted CCD data in the aforementioned 
15 compression means 31 A in each of the aforementioned embodiments 1 to 5, the 
difference between the values of pixel data is preferably calculated for compressing the 
data. Fig. 9A is a schematic diagram for illustrating differential compression according 
to an embodiment 6 of the present invention, and Fig. 9B is a schematic block diagram 
showing an exemplary structure implementing the differential compression. As shown 
20 in Fig. 9B, CCD data A/D converted in the aforementioned analog signal processing 
circuit 13 is input in difference calculation means 41, which in turn calculates the 
difference between the values of pixels adjacent to each other along the time base in each 
line forming a frame for converting the CCD data to differential data. As shown in Fig. 
9A, the difference calculation means 41 calculates the differences between pixel values 
25 of the same color components between pairs of pixels 40i and 402, 40 2 and 40 3 , 4O3 and 
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40 4 , - in pixels 40 h 40 2 , 40 3 , - forming CCD data 40 of a single line for generating 
differential data. Then, compression means 31A performs reversible coding such as 
entropy coding on such differential data, so that compressed data thereof is transferred to 
and stored in a raw image data buffer 26a of a main memory 26. 
Embodiment 7. 

Figs. 10A and 10B show another exemplary differential compression according 
to an embodiment 7 of the present invention. Fig. 10A is a schematic diagram for 
illustrating the differential compression according to the embodiment 7, and Fig. 1 OB is a 
schematic block diagram showing an exemplary structure implementing the differential 
compression. As shown in Fig. 10B, A/D converted CCD data is input in difference 
calculation means 44, which in turn calculates the difference between the values of 
alternate pixels along the time base in each line forming a frame for converting the CCD 
data to differential data. As shown in Fig. 10A, the difference calculation means 44 
calculates the differences between pixel values of the same color components between 
alternate pixels 40i and 40 3 , 40 2 and 40 4 , 40 3 and 40 5 , - in pixels 40i, 40 2 , 40 3 , - 
forming CCD data of a single line for generating differential data. Then, compression 
means 31 A performs reversible coding such as entropy coding on such differential data, 
so that compressed data thereof is transferred to and stored in a raw image data buffer 26a 
of a main memory 26. 

Embodiment 8. 

Figs. 11A and 11B show still another exemplary differential compression 
according to an embodiment 8 of the present invention. Fig. 11A is a schematic 
diagram for illustrating the differential compression according to the embodiment 8, and 
Fig. 1 IB is a schematic block diagram showing an exemplary structure implementing the 
differential compression. As shown in Fig. 11 A, difference calculation means 46 
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calculates the difference A between pixel values of the same color component as to 
vertically adjacent two pixels mapped by dotted lines between adjacent two lines LI and 
L2 and similarly calculates differences as to pairs of pixels between two lines L2 and L3, 
two lines L3 and L4, - thereby converting CCD data 40 to differential data. As shown 
in Fig. 11B, the difference calculation means 46 generates the differential data and 
compression means 31A compresses the differential data, which in turn is transferred to a 
main memory 26 and stored in a raw image data buffer 26a. 
Embodiment 9. 

Figs. 12A and 12B show further exemplary differential compression according 
to an embodiment 9 of the present invention. Fig. 12A is a schematic diagram for 
illustrating the differential compression according to the embodiment 9, and Fig. 12B is a 
schematic block diagram showing an exemplary structure implementing the differential 
compression. As shown in Fig. 12 A, difference calculation means 47 calculates the 
differences between pixel values of the same color components as to vertically adjacent 
pairs of pixels shown by dotted lines between alternate pairs of lines LI and L3, L2 and 
L4, L3 and L5, - thereby converting CCD data 40 to differential data. As shown in Fig. 
12B, the difference calculation means 47 generates the differential data and compression 
means 31 A compresses the differential data, which in turn is transferred to a main 
memory 26 and stored in a raw image data buffer 26a. 

Either the difference calculation means 46 according to the aforementioned 
embodiment 8 or the difference calculation means 47 according to the aforementioned 
embodiment 9 is preferably selected in response to the arrangement of a color filter array 
on CCD pixels and a driving system. The color filter array may be arranged in various 
ways, such that horizontal lines (odd lines) arranging color filters in order of R (red), G 
(green), R, G, - and horizontal lines (even lines) arranging color filters in order of G 
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(green), B (blue), G, B, ••• are alternately arranged in the vertical direction in the basic 
form of a Bayer system color filter array, for example. In this case, pixels of the same 
color component appear on alternate lines, and differential compression of pixel values of 
the same color component is enabled by selecting the difference calculation means 47 
5 according to the aforementioned embodiment 9 when the CCD 12 is driven by the 
aforementioned progressive system. When the CCD 12 having such a color filter array 
is driven by the aforementioned interlacing system, on the other hand, differential 
compression of pixels of the same color component is enabled by selecting the difference 
calculation means 46 according to the aforementioned embodiment 8 with respect to odd 

1 0 and even fields read as the CCD data respectively. 

While the compressed data stored in the raw image data buffer 26a is expanded 
in the expansion means 3 IB and thereafter output to the RPU 14 in each of the 
aforementioned embodiments 6 to 9, processing of decoding the expanded data 
(differential compression signal) expanded in the expanded means 3 IB to original data 

15 not yet subjected to differential compression is executed before executing the 
aforementioned real-time image processing in the RPU 14. 

While the invention has been shown and described in detail, the foregoing 
description is in all aspects illustrative and not restrictive. It is therefore understood that 
numerous modifications and variations can be devised without departing from the scope 

20 of the invention. 



